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Background 

A handful of colleges and universities sit as the uncrowned princes of the U.S. system of higher 
education. With enormous endowments, renowned faculty, and international prestige, there is no 
question that a comparative handful of institutions—most of them private—stand above the rest. 
These colleges are highly selective, such as the prestigious Ivy League schools, which have an 
average acceptance rate of under ten percent 1 . They attract of preponderance of the highest- 
achieving high school graduates, as evidenced by the average SAT scores of incoming freshman 
(Hoxby 2009). And, as many of the top schools are private, they tend to be much more expensive 
to attend: the total charges (including tuition and fees) for in-state four year public colleges in 
2010 was $16,000 per year, less than half of that of private non-profits at $37,000 (Baum and Ma 
2010 ). 

What is less clear is what these bright students and their families are getting for their 
money. At first blush this might seem a rather odd question. After all, students at elite 
institutions of higher education tend to graduate at a higher rate, are more likely to pursue 
advanced degrees, have more prestigious careers, and earn higher salaries than students at other 
universities (Camevale and Rose 2003). The disproportionate rewards accruing to students at top 
colleges have aroused a great deal of concern due to their underrepresentation of women, 
minorities, and those from families of modest means. However, simply because graduates of the 
best schools do rather well in life tells us little about the quality of education they have received. 
First, it should be remembered that education is in part a positional good (Hollis 1982): one need 
not leam a great deal, only more than one’s competitors. Higher incomes associated with having 
gone to a Harvard or Yale are not necessarily an indicator of having received an objectively 
excellent education, only that their graduates’ educations are deemed superior to that obtained by 
others elsewhere. Second, the very privileged status of those entering the most prestigious 
schools raises doubts about the rigor of the instruction provided. The success of those graduating 
from elite colleges could simply be an instance of selection bias: they attract the best students in 

1 Calculated from data presented in Dell (2011) 
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part through reputation and price signaling, so it should not be surprising that their alumni do 
rather well. These postsecondary institutions provide superior social networking advantages and 
peer effects, real benefits to be sure, but hardly indicators of a strong curriculum. There is every 
possibility that students at the most selective postsecondary institutions would do just as well 
having gone somewhere else. These considerations raise the question: is the selectivity of a 
college necessarily an indicator of the quality of education received? 

The difficulty to date in judging the quality of higher educational institutions is that the 
most common metrics are selectivity measures like acceptance rates, input measures such as 
SAT scores, and outcome measures such as wage and career data. What has been lacking thus far 
are measures of the value added by these schools. In other words, we need a way of determining 
how much additional knowledge has been gained by students in a given educational program in 
comparison with other programs. From the U.S. Teacher Education Study in Mathematics 
(TEDS), we now have such information. The TEDS yields empirical data on the educational 
experiences and content knowledge gains of a nationally representative sample of future teachers 
prepared at a wide variety of institutions. By combining data from the TEDS study with college 
selectivity measures—specifically the respected Barron’s college rankings—it is now possible to 
estimate the degree to which selective schools provide a superior education, at least in one 
academic discipline. Other studies, such as that of Arum, Roksa, and Choe (2011), examined 
learning gains across disciplines using generic measures of cognitive ability such as critical 
thinking that may or may not be appropriate to specific majors. The virtue of the TEDS study is 
that it focuses on only one—those preparing to become teachers of primary and early secondary 
mathematics—which permits specific assessments of content acquisition. In this paper we focus 
on future lower secondary teachers. 


Study Design & Preliminary Analysis 

The data for this research were gathered from two main sources: the Barron’s College 
Admissions Selector Rating, generously provided by Barron’s, and the results of the U.S.-TEDS 
study. By examining the relationship between student academic achievement and content 
knowledge with college selectivity, we can examine the value added by attending a given 
institution of higher education. We perform this analysis both at the institutional and student 
level. If “selectivity” truly does connote a higher quality education, students who attend elite 
schools should perform much better than students at other schools, controlling for the students ’ 
prior academic achievement. 

Probably the most commonly-used measure of college selectivity is the Barron’s index, 
published every year in the Barron’s Guide to the Most Competitive Colleges. (College Division 
of Barron's Education Series 2006) The index has been used by many researchers studying 


2 The use of the term “value added’’ is not to be confused with the quite different “value-added models’’ that are in 
widespread use as measures of school and teacher quality in U.S. K-12 education. 
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college selectivity. Barron’s has even partnered with the U.S. National Center on Educational 
Statistics to offer a publicly available longitudinal database. The Barron’s College Admissions 
Selector Rating classifies colleges into ten categories, with the most selective colleges receiving 
a rating of 10 and non-competitive schools a rating of 2, with specialty institutions assigned a 
coding of 1. Figure One presents the ten categories, with examples. 

Table 1: The College Admissions Selector Rating indicates the degree of competitiveness of 
admission to the college, with examples.* 


Barron's Rating 

Examples 

Most Competitive (10) 

Harvard, Northwestern University, University, of North 
Carolina-Chapel Hill 

Highly Competitive+ (9) 

University of California, Berkeley, University of Illinois 

Highly Competitive (8) 

Pennsylvania State University, University of Wisconsin 

Very Competitive-i- (7) 

Rochester Institute of Technology, University of Maryland 

Very Competitive (6) 

Michigan State University, Indiana University, Purdue 

Competitive+ (5) 

Illinois State University, University of Colorado - Boulder 

Competitive (4) 

San Diego State University 

Less Competitive (3) 

California State University 

Non-Competitive (2) 

University of Toledo 

Special (1) 

New England Conservatory of Music 


*College Division of Barron's Education Series (Ed.). (2006). Barron's Profiles of American Colleges: 2007 (27th 
ed.). Hauppauge, NY: Barron's Educational Series, Inc., page 252 


Sponsored by the International Association for the Evaluation of Educational 
Achievement (IEA), the same organization that conducts the TIMSS study, the TEDS was 
conducted as a follow-up to the MT21 study in multiple countries, including the United States. ( 
Schmidt et al. 2011) Nine hundred future lower secondary teachers in their final year of 
preparation at eighty-one postsecondary schools completed a questionnaire about their personal 
backgrounds, pre-collegiate educational experiences (including SAT math scores 3 ), the types of 
coursework and field experience they received at their preparatory institution, and their beliefs 
about and attitudes towards teaching mathematics. They also took the lower secondary version of 
the Future Teacher Mathematical Content Knowledge (MCK) test assessing their knowledge of 
mathematical topics on key domains. Additional surveys were directed towards institutional 


3 Some institutions reported only ACT scores, which were transformed into equivalent SAT scores. 
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administrators and faculty to glean information about the course curriculum and minimum 
requirements, among other items. 

Although the precise metric employed in the Barron’s Selector Rating is privileged 
information, it is quite likely that the SAT scores of incoming freshmen are an important 
component. In Figure One we present a scatterplot with the standardized SAT score on the x-axis 
and the 2 through 10 Barron’s rating on the y-axis (“special” schools coded 1 are excluded from 
our analysis because they do not prepare teachers), with each plot representing mean of each 
institution. This spatial representation confirms that there is a relationship between the selectivity 
of the school according to the Barron’s ranking and the quality of the student population, at least 
among those who intend to become lower secondary mathematics teachers. The relationship is a 
modest one, however, with a bivariate correlation of .48, which could be because the Barrons 
ranking presumably considers the quality of the entire student body, not just future teachers. It is 
somewhat surprising that the relationship between institutional ranking and incoming SAT 
scores isn’t higher, given how fierce the competition is for slots at elite colleges. 4 


4 One possibility is that those entering teacher preparation programs in less competitive schools have higher SAT 
scores relative to their institution, while those preparing to become teachers at competitive schools have lower 
relative SAT scores. 
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Figure 1. Barron’s Selector Rating vs. SAT 
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There is also a relationship between the Barron’s ranking and the quality of the student 
body exiting postsecondary institutions, as measured by the mathematical content knowledge of 
future teachers who are close to graduating from lower secondary teacher preparation programs. 
This relationship is presented in Figure Two, with a modest correlation between institutional 
ranking and performance on the MCK test of .40. On the surface these results might lead one to 
believe that more selective schools do in fact produce better-trained future teachers. However, 
because there is every reason to expect that students who perform well on the SAT would also 
receive a good score on the MCK. The apparently superior performance of students from 
prestigious schools might therefore be a case of selection bias. 
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Figure 2. Barron’s Selector Rating vs. MCK 



Selectivity and Program Quality 

A more accurate measure of the educational rigor of a given program can be found by comparing 
the performance of an institution’s students on the SAT and the MCK. SAT mathematics scores 
serve as the measure of a student’s preparation in mathematics before they enter a teacher 
preparation program, while the MCK score is a measure of a student’s knowledge near the 
completion of that program. If an institution’s students perfonn relatively better on the MCK 
than on the SAT, then we can infer that the teacher preparation program added a something to its 
future teachers’ store of mathematical knowledge. However, if student inputs (SAT) and outputs 
(MCK) are equivalent, then we should question how much added value that program provides. 
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Figure 3 


SAT vs MCK By Barron's Ranking 



Figure Three relates student performance on the MCK to scores on the math portion of 
the SAT, with both measures standardized so that they are on the same scale. The plots 
themselves are displayed in nine different shapes representing the nine different levels on the 
Barron’s selectivity scale. Open shapes indicate more selective schools and closed shapes less 
selective schools. Since both the SAT and MCK are measures of mathematical knowledge, there 
is a strong correlation between the two (.78)—students who do well on one test tend to do well 
on the other test. The forty-five degree line in the center of the figure represents an exact 
correspondence in the performance of an institution’s students on the standardized SAT and 
MCK—in other words, that on average students demonstrated no relative gains in mathematical 
knowledge during their teacher preparation program. The further above the line an institution is, 
the greater the relative average learning gains for its students. As is clear from the graph, the 
average student at most of the eighty-one institutions in the sample registered learning gains in 
mathematics content knowledge during their college years 5 . However, there is no clear 


5 Part of this increase may be due to differing sample populations. The SAT is normed on the U.S. population, while 
the MCK is normed internationally. As the U.S. average on the MCK is slightly higher than the average SAT math 
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relationship between the selectivity of the institution and relative improvement on assessments 
mathematics content knowledge. There are a substantial number of institutions ranked low on the 
Barron’s index which nevertheless are located above the line, and some of the more selective 
institutions fall below the line, indicating that their students actually performed less well relative 
to comparable students at other schools after four years of college education. 

The quadrant in which the institution is plotted also reveals information about the quality 
of the students attending each type of college or university. Those institutions in the top right 
quadrant recruit high-quality students who also do well on the MCK, with those above the line 
posting increases in relative performance. Although the highest-performing institutions are 
relatively selective, there are nearly as many lower-ranked schools that also recruit bright 
students who exhibiting greater relative performance on the MCK. 

Given their selectivity, why aren’t the average SAT scores at selective schools not all 
grouped in the upper right quadrant? And why isn’t there a stronger relationship between 
selectivity and SAT scores? Although we can only speculate, it is possible that, of those 
interested in becoming teachers, the “best and brightest” students as measured by SAT scores are 
not attracted to more selective schools. Given their great expense, students instead choose to 
attend less selective state public schools. State schools are designed with state teaching 
credentials in mind, which might not be the case for private schools. Additionally, the salary for 
teachers is low compared with that in other professions that have comparable educations. As a 
consequence it might make little sense from a strict cost-benefit perspective to obtain a very 
expensive degree from a private institution. As this study is focused only on future lower 
secondary mathematics teachers, the results should not be analogized to other disciplines, or to 
any class of universities as a whole. There is every possibility that the Barron’s rankings are 
more strongly correlated to SAT scores for the whole student body, or that different programs at 
elite schools might demonstrate larger increases in relative performance. 

Because of the small number of institutions in the sample and to reduce the number of 
interaction terms, we collapsed the competitiveness ratings into four broader categories: Level 4 
Most Selective (Most Competitive and Highly Competitive +), Level 3 Very Selective (Highly 
Competitive, Very Competitive +, and Very Competitive), Level 2 Selective (Competitive + and 
Competitive), and Level 1 Less Selective (Less Competitive and Non-Competitive). Table 2 
presents a simplified representation of Figure 3 indicating the percentage of institutions in each 
category, with “above” referring to institutions whose average performance is greater than the 
baseline, which suggests that students attending those institutions have increased their relative 
knowledge of mathematics, and “on/below” referring to those institutions whose students have 
experienced no increase or have even fallen behind their peers at other institutions. 


score, part of the difference in performance between the two exams is based on that higher average. This likely 
contributes only to a modest proportion of the overall score gain, however. 
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Table 2. Quality Indicator by Simplified Barron’s Selector Categories 



Level 1: 

Less 

Selective 

Level 2: 

Selective 

Lever 3: 

Very 

Selective 

Level 4: 

Most 

Selective 

Above Quality Baseline 

16 

41 

32 

11 

On/Below Quality Baseline 

19 

39 

32 

10 


(cells indicate number of institutions in each category) 


If we define a “high-quality” teacher preparation program as one whose students score 
more highly than their entrance examination scores would predict (and “low-quality” as those 
whose relative performance is lower), then it becomes clear that there is very little relationship 
between the college selectivity and program quality, at least in the field of lower secondary 
mathematics education. At each level of selectivity, a roughly equal proportion of institutions are 
above the baseline as on or below it. 

An analysis of institutional data drawn from the US-TEDS study suggests that there are 
important differences between “high-quality” and “low-quality” teacher education programs, 
independent from institutional selectivity. In previous work (Schmidt et al. 2002, Schmidt et al. 
2011) we have argued that a rigorous mathematics curriculum is essential for learning 
mathematics for both students and future teachers. As indicated in Figure 4, high-quality teacher 
education programs—those whose students MCK scores are much greater than their SAT math 
scores—share a common emphasis on mathematics coursework. Although the proportion of 
required math courses increases with college selectivity, the difference between high-quality and 
low-quality selective institutions in the number of math courses is particularly pronounced. The 
main dividing line in program quality is not school selectivity. There also appears to be a 
tendency for higher-quality but low selectivity institutions to focus more on general studies 
course requirements, while high-quality and high-selectivity institutions emphasize mathematics 
classes (see Figure 5). 
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Figure 5. Required Course Hours in General Studies by Barron’s Selector Categories 
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The last stage of our examination of the relationship between college selectivity and 
college quality in lower secondary mathematics teacher preparation programs is a more rigorous 
statistical analysis using multi-level modeling techniques. The data is drawn from the US-TEDS 
institutional and student survey and includes both individual level variables measuring previous 
academic coursework (high school mathematics coursework and score on the SAT) and 
postsecondary training, as well as program-level indicators of course requirements, selectivity, 
and mean SAT scores (see Table 4). The dependent variable is the student’s score on the MCK. 
The interaction terms are formed from two categorical variables: the simplified Barron’s ranking 
of selectivity and whether the teacher preparation program was classified above or on/below the 
line of equal input and output. High-selectivity institutions whose students perform above the 
baseline serve as the reference category. If selectivity is a proxy for quality, then we should 
expect school selectivity to relate to individual-level achievement (although it is clear from 
Figure 3 that not all selective schools are “high-quality” in terms of improved relative 
performance). Alternatively, a strong impact from specific institutional features (such as 
coursework requirements) or average school outcomes (defined as being above or below the 
baseline) would imply that the design of the teacher program is related to student knowledge 
gains, i.e. that curriculum matters. Finally, stronger effects due to student characteristics (the 
average SAT score) would suggest that students who know relatively more entering teacher 
preparation programs gain relatively more knowledge by the time they graduate. 
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Table 3 Predicting MCK by Selection & Course Indicators 


Source 

Est 

(se) 

P< 

Intercept 

193.54 

57.28 

0.001 

Future Teacher Level* 

Highest Mathematics Course Taken in High School 

11.87 

1.97 

<.0001 

College Entrance Mathematics Score 

0.14 

0.02 

<.0001 

Advanced Mathematics OTL Exposure 

5.31 

1.27 

<.0001 

Program Level 

College Entrance Mathematics Score 

0.53 

0.09 

<.0001 

Number of Required Math Course Hours 

0.04 

0.01 

0.0119 

Advanced Mathematics OTL Exposure 

12.63 

4.82 

0.0101 

Less Selective & At/Below Baseline 

-41.60 

17.54 

0.0196 

Less Selective & Above Baseline 

-10.32 

18.15 

0.5711 

Selective & At/Below Baseline 

-43.83 

15.48 

0.0056 

Selective & Above Baseline 

-21.61 

16.94 

0.205 

Very Selective & At/Below Baseline 

-49.27 

14.72 

0.0011 

Very Selective & Above Baseline 

-30.15 

15.47 

0.054 

Most Selective & At/Below Baseline 

-47.73 

19.5 

0.0161 

Most Selective & Above Baseline 

Reference category 



* Variables centered on program means. 


The results of the multi-level model are presented in Table 3. Predictably, individual 
student background is strongly associated with performance on the MCK, with students taking 
more advanced high school mathematics courses and with higher SAT scores earning better 
MCK results. This is an individual level relationship which holds across all institutions. Most 
relevant for the present study, the only variables that fail to achieve statistical significance are the 
interactive terms for selective, high quality institutions (those that are on average above the 
baseline). Students attending institutions whose future teachers on average experienced relative 
knowledge gains in mathematics did no better if they went to a selective school, controlling for 
other factors. Similarly, future teachers at low-quality institutions did about the same no matter 
how selective their institution, performing forty to fifty points worse on the MCK than those at 
high-quality, high-selectivity schools, again controlling for other factors. 
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Each of the other institutional characteristics had a statistically significant and positive 
relation with student MCK scores. Students at institutions with higher average SAT scores 
tended to receive a higher MCK score. The curriculum of the teacher preparation program was 
also associated with better outcomes on the MCK, with both required number of course hours in 
mathematics and the average hours in advanced mathematics courses taken by future teachers 
demonstrating a statistically significant relationship. Students with more coursework also tended 
to have higher scores. 

One key outcome of our analysis is that institutional and program-level factors appear to 
play a greater role than individual-level characteristics. Although they are statistically 
significant, the coefficients for student-level SAT performance and number of advanced math 
courses are substantially smaller than the coefficients representing the institutional mean for SAT 
or advanced math coursework. The association with MCK scores was more than twice as great 
for the institutional average in the number of advanced courses than individual student behavior 
(12.63 vs. 5.31), while coefficients for mean SAT scores were more than triple that of individual 
student scores on the math section of the SAT (.54 vs. .14). We can only speculate precisely how 
institutional average SAT scores influence individual MCK scores, for example through peer 
effects or the ability to devote more time and resources to advanced preparation. More clear is 
that curriculum and institutional design is strongly related to how well future teachers are 
grounded in mathematics. 


Conclusion 

The top schools in the U.S. hold their status for a reason, not least of which is that students and 
faculty compete fiercely to become part of them. However, rankings such as those generated by 
U.S. News & World Report or Barron’s tell us more about the reputations of those schools than 
about their ability to deliver a high-quality education. Any easy attributions about the worth of a 
postsecondary institution are confounded by the problem of selection bias. 

In the instance of teacher education this problem is particularly severe, with 
consequences both for prospective teachers and for the educational system as a whole. Starting 
teachers in the United States make approximately 80% of per capita income, much less than 
similarly educated professions. Attending an expensive private institution makes little sense 
unless it comes with a commensurate reward. Our study suggests that in many cases those 
interested in teaching mathematics might be better served by attending less expensive, less 
exclusive public institutions. More importantly, the greater importance placed upon teacher 
quality by policymakers makes it imperative to identify the best models of teacher preparation, 
most especially in mathematics. Our analysis makes it clear that we will not be able to find those 
models simply by examining the course requirements at Ivy League institutions. We will need to 
do the careful work of examining a broad range of institutions, with close attention paid not to 
what sort of students attend a program but what sort of teachers they are when they graduate. 
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